Palij, 95, was born in what was then-Poland and now Ukraine, and immigrated to the United States in 1949.
The former Nazi guard apparently lied to U.S. immigration officials about his role in mass killings during World War II, saying he worked on a farm and in a factory, the White House said in a statement.
The former Nazi labor camp guard has lived in Jackson Heights, Queens since 2001.
His deportation comes after years of diplomatic wrangling, the White House announced on Tuesday.
Palij became a U.S. citizen in 1957 and has been living a relatively solitary life for decades in New York City despite past failed efforts to deport him, according to media reports.
U.S. Rep. Nita Lowey, D-Harrison, who represents much of Westchester as well as Rockland County and is the ranking Democrat on the House Appropriations Committee and its Subcommittee on State and Foreign Operations, applauded the move.
“Jakiw Palij oversaw untold atrocities against innocent men, women, and children at the Nazi Trawniki camp," Lowey said. "His presence in New York has served as a painful reminder and insult to those who fought in World War II or lost loved ones in the Holocaust."
"Today’s news of Palij’s deportation is a win for justice and an affirmation of the United States’ opposition to hate. I appreciate U.S. government officials’ long-time work on this important issue and the German government’s willingness to accept the deportation," Lowey said.
Last year, Lowey helped secure an amendment to the Fiscal Year 2018 Appropriations bill to urge the State Department to "speedily deport" Palij.
The White House statement added: "By serving as an armed guard at the Trawniki Labor Camp and preventing the escape of Jewish prisoners during his Nazi service, Palij played an indispensable role in ensuring that the Trawniki Jewish victims met their horrific fate at the hands of the Nazis."
Palij repeatedly denied wrongdoing in court papers, claiming that he and other young men from his Polish hometown were coerced into working for the Nazi occupiers.
In 2003, Palij's American citizenship was revoked. In 2004, a federal judge ordered that Palij be deported -- but none of the European countries to which he could have been sent, would take him.
In a statement on Tuesday, U.S. Attorney General Jeff Sessions praised the work of the Justice Department's best-known Nazi hunter, Eli Rosenbaum, and his team in successfully removing the 68th Nazi from the U.S.
Palij's case represents the closing of an era -- until now he was the only remaining active case from the Nazi era pursued by the Justice Department's Office of Human Rights and Special Prosecutions.
The atrocities of the Trawniki camp, where Palij worked, aren't well known because the killing and coverups were thorough, historians now say. One document mentioned an operation that killed 4,000 people at Trawniki, mostly Jews.
Click here to follow Daily Voice New City and receive free news updates.